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Abstract. Stochastic thermodynamics provides a framework for describing small systems like colloids or 
biomolecules driven out of equilibrium but still in contact with a heat bath. Both, a first-law like energy 
balance involving exchanged heat and entropy production entering refinements of the second law can 
consistently be defined along single stochastic trajectories. Various exact relations involving the distribution 
of such quantities like integral and detailed fluctuation theorems for total entropy production and the 
Jarzynski relation follow from such an approach based on Langevin dynamics. Analogues of these relations 
can be proven for any system obeying a stochastic master equation like, in particular, (bio)chemically 
driven enzyms or whole reaction networks. The perspective of investigating such relations for stochastic 
field equations like the Kardar-Parisi-Zhang equation is sketched as well. 



PACS. 05.40.-a Fluctuation phenomena, random processes, noise, and Brownian motion 
modynamics 



05.70.-a Ther- 



1 Introduction 

Stochastic thermodynamics as a conceptual framework 
combines the stochastic energetics approach introduced a 
decade ago by Sekimoto [1] with the idea that entropy can 
consistently be assigned to a single fluctuating trajectory 
[2]. Stochastic energetics has been developed for meso- 
scopic systems like colloidal particles or single (bio)molecules 
driven out of equilibrium by time-dependent forces but 
still embedded in a heat bath of well-defined tempera- 
ture. The dynamics of the few degrees of freedom are mod- 
elled by a Langevin equation where the surrounding heat 
bath provides the thermal noise. A first-law-like energy 
balance relates applied or extracted work, exchanged heat 
and changes in internal energy along the single fluctuat- 
ing trajectory. In the ensemble, these quantities acquire 
no longer sharp values but distributions. Since dissipated 
heat is typically associated with changes in entropy this 
program of using the classical thermodynamic notions to 
describe the trajectories of these small systems obviously 
requires also the concept of a fluctuating entropy with 
corresponding distribution in the ensemble. 

Stochastic thermodynamics being based on stochastic 
equations of motion allows also to embed and categorize 
two recent developments orginally derived independently 
using different equations of motion. First, the (detailed) 
fluctuation theorem dealing with non-equilibrium steady 
states provides a symmetry between the probability to ob- 
serve asymptotically a certain entropy production and the 
probability for the corresponding entropy annihilation [3- 
6] . Second, the Jarzynski relation expresses the free energy 
difference between two equilibrium states as a non-linear 
average over the non-equilibirum work required to drive 



the system from one state to the other in a finite time 
[7-10]. Within stochastic thermodynamics both of these 
relations can easily be derived and the latter shown to be 
a special case of a more general relation [2] . 

The purpose of this article is to describe the princi- 
ples of stochastic thermodynamics in a systematic way 
and to sketch a few perspectives. By character mainly a 
concise review (with a few original results) from a per- 
sonal unifying perspective, no attempt is made to achieve 
a comprehensive historical presentation. Several (mostly 
review) articles can provide complementary and occasion- 
ally broader perspectives [11-25]. Focussing on the the- 
oretical concepts I will point to experimental work only 
where these experiments serve as an illustration of the 
theory. Refererence to the growing number of experiments 
and numerical simulations where these concepts are used 
to learn details about particular systems can also be found 
in those articles. 



2 Stochastic energetics 

2.1 Stochastic Langevin dynamics 

As basic model, we consider overdamped motion x(t) of a 
"particle" or "system" governed by the Langevin equation 



x = fxF(x, A) + C 



(1) 



where F(x, A) is a systematic force and £ thermal noise 
with correlation {({t)((t')) — 2DS(t — r') where D is the 
diffusion constant. In equilibrium, D and the mobility /j. 
are related by the Einstein relation D = Tfj, where T is the 
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temperature of the surrounding medium with Boltzmann's 
constant set to unity throughout the paper to make en- 
tropy dimensionless. In stochastic thermodynamics, on as- 
sumes that the strength of the noise is not affected by the 
presence of a time-dependent force. The range of validity 
of this crucial assumption can be tested experimentally 
or in simulations by comparing with theoretical result de- 
rived on the basis of this assumption. 
The force 



F(x,X) = -d x V{x,X)+f(x,X) 



(2) 



can arise from a conservative potential V(x, A) and/or be 
applied to the particle directly as f(x,X). Both sources 
may be time-dependent through an external control pa- 
rameter A(t) varied according to some prescribed experi- 
mental protocol from A(0) = Ao to X(t) = X t . 

To keep the notation simple, we treat the coordinate x 
as if it were a single degree of freedom. In fact, all results 
discussed in the following hold for an arbitrary number 
of coupled degrees of freedom for which x and F become 
vectors and D and /x (possibly x-dependent) matrices. 

Equivalent to the Langevin equation is the correspond- 
ing Fokker-Planck equation for the probability p(x, r) to 
find the particle at x at time r as 

d T p(x,r) = -d x j(x,r) 

= -d x (fJ,F(x, X)p(x, t) - Dd x p(x, r)) (3) 

where j(x, r) is the current. This partial differential equa- 
tion must be augmented by a normalized initial distribu- 
tion p(x, 0) = po{x). It will become crucial to distinguish 
the dynamical solution p(x, r) of this Fokker-Planck equa- 
tion, which depends on this given initial condition, from 
the solution p s (x, A) for which the rhs of eq. (3) vanishes 
at any fixed A. The latter corresponds either to a steady 
state for a non-vanishing non-conservative force / ^ or 
to equilibrium for / = 0, respectively. 

A third equivalent description of the dynamics is given 
by assigning a weight 



p[x(t)\x ] = cxp 
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dr[(x - fiF) 2 /AD + fiF' /2] 



(4) 



to each path or trajectory. The last term in the exponent 
arises from the Jacobian \d(/dx\ in the Stratonovitch dis- 
cretization required because the weight for a noise his- 
tory p[C(t)] is transformed into the weight of a trajectory 
p[x(r)]. Path dependent obscrvablcs can then be averaged 
using this weight in a path integral which requires a path- 
independent normalization such that summing the weight 
(4) over all paths is 1. 



2.2 First law 

As a major ingredient to stochastic thermodynamics, the 
first-law-like energy balance 



identified for a Langevin equation by Sekimoto within his 
stochastic energetics approach [1] needs to be recalled. 
The increment in work applied to the system 

dw = {dV/dX) Xdr + f dx (6) 

consists of two contributions. The first term arises from 
changing the potential (at fixed particle position) and the 
second from applying a non-conservative force to the par- 
ticle directly. If one accepts these quite natural definitions, 
for the first law to hold along a trajectory the heat dissi- 
pated into the medium must be identified with 

dq = Fdx. (7) 

This relation is quite physical since in an overdamped sys- 
tem the total force times the displacement corresponds to 
dissipation. Integrated along a trajectory of given length, 
one obtains the expression 

w[x(t)] = q[x(T)} +AV = q[x(r)} + V(x t , X t ) - V(x , A ) 

(8) 

It is interesting and crucial to note that the heat dis- 
sipated along a trajectory can also be written using the 
path integral weight as 

where p[x (r) |io] is the weight of the backward path x(t) = 
x(t — t) starting at x(0) = x{t) under the time-reversed 
protocol A(r) = X(t — t). 

In a recent experiment [26], the three quantities ap- 
plied work, exchanged heat and internal energy were in- 
ferred from the trajectory of a colloidal particle pushed 
periodically by a laser trap against a repulsive substrate. 
The measured non-Gaussian distribution for the applied 
work shows that this sytem is driven beyond the linear re- 
sponse regime since it has been proven that within the lin- 
ear response regime the work distribution is always Gaus- 
sian [27]. Moreover, the good agreement between the ex- 
perimentally measured distribution and the theoretically 
calculated one indicates that the assumption of noise cor- 
relations being unaffected by the driving is still valid in 
this regime beyond linear response. 



3 Entropy production 

3.1 Entropy production in the medium 

The heat dissipated into the environment should be iden- 
tified with an increase in entropy of the medium 



As m [x(T)]=q[x(T)]/T. 



(10) 



dw — dV + dq 



(5) 



From a purely classical perspective, it might be disturb- 
ing to define an entropy change which depends on the 
trajectory and thus becomes a stochastic quantity. Usu- 
ally, entropy is considered to be an ensemble quantity. In 
fact, it turns out that one can and should go even one 
step further and assign a second contribution to entropy 
as follows. 
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3.2 System entropy 

Having solved the Fokker-Planck equation (3), one defines 
as a trajectory dependent entropy of the system the quan- 
tity [2] 

S (T) = -\np(x(r),T) (11) 

where the probability p(x, r) obtained by solving the Fokker- 
Planck equation is evaluated along the stochastic trajec- 
tory x(t). Obviously, for any given trajectory x{t), the 
entropy s(t) depends on the given initial data po(x) and 
thus contains information on the whole ensemble. What 
at first sight might look just as a definition becomes useful 
and interesting through the following observations: 

(i) Upon averaging with the given ensemble p(x, r), the 
trajectory-dependent entropy becomes the usual ensemble 
entropy 



p(x, t) is a density. Apparently more disturbingly, this ex- 
pression is not invariant under non-linear transformations 
of the coordinates. In fact, both deficiencies which also 
hold for the ensemble entropy (12) are related and can be 
cured easily as follows by implicitly invoking the notion of 
relative entropy. 

A formally proper definition of the stochastic entropy 
starts by describing the trajectory using canonical vari- 
ables. After integrating out the momenta, for a system 
with N particles with Cartesian positions {xi}, one should 
define the entropy as 



s({xi(r)}) 



ln\p({ Xi (r)})X 3 T N } 



(18) 



where At is the thermal de Broglie length. If one now con- 
siders this dynamics in other coordinates {yi}, one should 
use 



S(t) = - J dx P {x,t) In P (x,t) = (s(r)>. (12) = - ^({^(r)}, r) det{dy/dx}X^ N ] 



(19) 



(ii) In equilibrium, i.e. for / = and constant A, this 
definition assigns a stochastic entropy 



s(x(t)) = (V(x(t),\)-F(\))/T, 



(13) 



with the free energy 

JT(A) = -Tin J dx exp[-V(x,X)/T]. (14) 

Thus, in equilibrium the well-known thermodynamic rela- 
tion now holds along the trajectory at any time. 

(iii) Most significantly, the total entropy change along 
a trajectory 

As tot = As m + As (15) 

with 

As = -lnp(x t , \ t ) + lnp(x , A ) (16) 
now obeys an integral fluctuation theorem (IFT) [2] 

{ e - As ^) = 1 (17) 

which can be interpreted as a refinement of the second 
law (Z\s to t) > 0. The latter follows from (17) by Jensen's 
inequality. Here and throughout the paper the brackets 
(...) denote the non-equilibrium average generated by the 
Langevin dynamics from some given initial distribution 
p(x,0) = p (x). 

This integral fluctuation theorem for Astot is quite 
universal since it holds for any kind of initial condition 
(not only for po(x ) = P s ( x o> ^o))) an y time-dependence 
of force and potential, with (for / = 0) and without (for 
/ 7^ 0) detailed balance at fixed A, and any length of tra- 
jectory t. 

3.3 Invariance under coordinate transformations 

The entropy as defined in (11) has the formal deficiency 
that strictly speaking h\p{x(r),T) is not defined since 



This correction with the Jacobian ensures that the en- 
tropy both on the trajectory as well as on the ensemble 
level is independent of the coordinates used to describe the 
stochastic motion. Of course, this statement is no longer 
true if the transformation from {x^ to {yi} is not one to 
one. Indeed, if some degrees of freedom are integrated out 
the entropy does and should change. 



4 A unifying integral fluctuation theorem 
(IFT) 

The IFT for entropy production (17) follows from a more 
general fluctuation theorem which unifies several relations 
previously derived independently. Based on the concept of 
time-reversed trajectories and time-reversed protocol [5, 
10,15], it is easy to prove the relation [2] 



(exp[-As m ] pi(x t )/po(x )) = 1 



(20) 



for any function p\{x) with normalization J dx pi(x) = 1. 
Here, the initial distribution po(x) is arbitrary. By using 
the first law (8) this relation can also be written in the 
form 

<exp[-(tu - AV)/T] pi(x t )/p (x )) = 1 (21) 

with no reference to an entropy change. 

The arguably most natural choice for the function p\ (x) 
is to take the solution p(x, r) of the Fokker-Planck equa- 
tion at time t which leads to the IFT (17) for the total 
entropy production. Other choices lead to the following 
relations originally derived differently. 



4.1 Jarzynski relation (JR) 

The Jarzynski relation [7] 

(exp[-w/T\) - (exp[-AF/T]> (22) 
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expresses the free energy difference AF = J-(X t ) — .F(Ao) 
between two equilibrium states as a non-linear average 
over the work required to drive the system from one equi- 
librium state to another. In the present formalism it fol- 
lows for motion in a time-dependent potential V(x, A(r)) 
without any additional non-conservative force / = by 
plugging into the general FT (21) the equilibrium initial 
distribution pa(x) = exp[— (V(x, Ao) — J-(Xq))/T] and the 
function pi(x) = exp[— (V(x, A t ) — T{X t ))/T\. It should 
be clear that this choice for p\ (x) exploits a mathematical 
freedom. It does not imply that the system has acquired 
this new equilibrium distribution at the end of the process. 
In fact, the actual distribution at the end will be p(x, t). 

4.2 Bochkov-Kuzovlev relation (BKR) 

For a system initially in equilibrium in a time-independent 
potential Vq{x), which is for < r < t subject to an addi- 
tional space and time-dependent force f(x, r), one obtains 
from (21) the relation 

(exp[-w/T}) = 1 (23) 

with 

w = [ ' f(x,X(T))dx (24) 

by choosing p x (x) = p (x) = exp[-(V (x) - Fq)/T]. Un- 
der these conditions, the exponent w is the work per- 
formed at the system. Since this relation derived much 
earlier by Bochkov and Kuzovlev [28,29] looks almost like 
the Jarzynski relation there have been both claims that 
the two are the same and some confusion around the ap- 
parent contradiction that exp[— w/T] seems to be both 
exp[-AF/T]) or 1. 

The present derivation shows that the two relations are 
different since they apply a priori to somewhat different 
situations. The JR as discussed above applies to processes 
in a time-dependent potential, whereas the BKR relation 
as discussed here applies to a process in a constant poten- 
tial with some additional force. If, however, in the latter 
case, this explicit force arises from a potential as well, 
f(x,r) = —V{(x,t), there still seems to be an ambiguity. 
It can be resolved by reckognizing that in this case the 
work entering the BKR (23) 

w = j ' dxf ' = - J dxV{{x) = -AVi + w (25) 

differs by a boundary term from the definition of work w 
given in eq. (6) and used throughout this paper. Thus, 
if the force arises from a time-dependent but conservative 
potential both the BKR in the form (exp[— w/T]) — 1 and 
the JR (22) hold. The connection between the two rela- 
tions has previously been discussed within a Hamiltonian 
dynamics approach [30]. 

4.3 Hummer and Szabo relation 

As an important application, the Jarzynski relation can 
been used to reconstruct the free energy landscape of a 



biomolecule G(x) where x denotes a "reaction coordinate" 
like the end-to-end distance in forced protein folding as 
reviewed in [19]. If one end of such a protein is attached 
to an AFM tip or an optical trap via a potential (k/2)(x — 
A(t)) 2 where k is the stiffness of the cantilever or trap and 
A(t) the time-dependent center, the total potential reads 

V(x, A(r)) = G(x) + (fc/2)(A(r) - xf . (26) 

The initial distribution p (x) = exp[— (V(x, A )— !F(Xo))/T] 
and the choice 

Pl (x) = S(x - z) cxp -[V(x, \ t )/T]/ exp -[V(z, X t )/T] 

(27) 

plugged into (20) leads to the potential reconstruction for- 
mula 

e- G ^ = (S[z - x{t)]e~ w V) e w*)(z-mr e H*o) (28) 

first derived by Hummer and Szabo [31] using a Feynman- 
Kac approach. Thus to get the potential G(z) it is suf- 
ficient to select those trajectories that have reached z 
after time t and record the corresponding work w(t) = 
k J* dr(A(r) — x(t)) accumulated up to time t. An exper- 
iment on unfolding RNA has been one of the first real- 
world test of this z-resolved Jarzynski relation [32]. 

4.4 Integral "end-point" relations 

A variety of "end-point" relations can be generated from 
(20) as follows. 

By choosing pi(x) — p(x, t)g(x)/(g(x t )), one obtains 

(g(x t )exp[-As tot ]) = (g(x t )) (29) 

for any function g(x) [33]. Likewise, for / = and V(x, A(r)), 
by choosing p x (x) = g(x) exp[-V(x, A t )-J/ r (A t )]/(ff(a;))e q ,A t , 
one obtains 

(g(x t ) exp[-{w - AT)/T]) = (g(x)) m , Xt (30) 

which has been first derived by Crooks [10]. Here, the 
average on the right hand side is the equilibrium average 
at the final value of the control parameter. 
In the same fashion, one can derive 

(g(x t )exp[-w/T\) = {g(x)) cqAo (31) 

by choosing pi(x) = g(x) cxp[-V(x, Ao)-J c "(A )]/(.g(x)} oq!Ao 
for a time-independent potential and arbitrary force f(x,r) 
which is the end-point relation corresponding to the BKR. 



5 Non-equilibrium steady states (NESSs) 
5.1 Characterization 

Non-equilibrium does not necessarily require that the sys- 
tem is driven by time-dependent potentials or forces as 
discussed so far. A non-equilibrium steady state (NESS) is 
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generated if time-independent but non-conservative forces The DFT for total entropy production holds even un- 

f(x) act on the system. Such systems are characterized by der the more general situation of periodic driving F(x, r) = 

a time-independent or stationary distribution F(x, t + t p ), where t p is the period, if (i) the system has 

settled into a periodic distribution p{x, r) = p(x, t + r p ), 

p s (x) ee exp[-0(ar)]. (32) and (ii) the trajectory length t is an integer multiple of 

T p . 

As a fundamental difficulty there seems to be no simple For the distribution of work p(W), a similar DFT can 

way to calculate p s {x) or, equivalently, the "non-equilibrium be provcn provided the protocol is symmetric A(r) = 

potential" <j>(x). In one dimension, it follows from quadra- X (t _ T ^ thc non-conservative force zero, and the sys- 

tures but for more degrees of freedom, setting the right- tems starts in equilibrium initially. For such conditions, 

hand-side of the Fokker-Plank equation (3) to zero repre- the DFT for work was reC ently tested experimentally us- 

sents a formidable partial differential equation. Physically, ing a a colloidal particle pushed periodically by a laser 

the complexity arises from the presence of a non-zero sta- trap against a repulsive substrate [26]. 
tionary current (in the full configuration space) 



f (x) = ^F(x)p s (x) - Dd xP s (x) ee v s (x)p s (x) (33) 

which defines the mean local velocity v s {x). This current 
leads to a mean entropy production rate 

o ee (As tot )/t = J dxfD-'f/f. (34) 

Even though the stationary distribution and currents can 
not be calculated in general, a variety of exact relations 
concerning entropy production and heat dissipation have 
been derived. 



5.2 Detailed fluctuation theorem (DFT) 

In a NESS, the (detailed) fluctuation theorem 

p(-As tot )/p(As t ot) = exp[-Zis tot ] (35) 

expresses a symmetry which the probability distribution 
p(Astot) for the total entropy production after time t in 
the steady state obeys. This relation has first been found 
in simulations of two-dimensional sheared fluids [3] and 
then been proven by Gallavotti and Cohen [4] using as- 
sumptions about chaotic dynamics. A much simpler proof 
has later been given by Kurchan [5] and Lebowitz and 
Spohn [6] using a stochastic dynamics for diffusive mo- 
tion. Strictly speaking, in all these works the relation holds 
only asymptotically in the long-time limit since entropy 
production had been associated with what is here called 
entropy production in the medium. If one includes the en- 
tropy change of the system (11), thc DFT holds even for 
finite times in the steady state [2]. This fact shows again 
the advantage of the entropy definition introduced above. 

While the DFT for (medium) entropy production has 
been tested experimentally for quite a number of systems, 
a first test including the system entropy has recently been 
achieved for a colloidal particle driven by a constant force 
along a periodic potential [34]. This experimental set-up 
constitutes the simplest realization of genuine NESS. The 
same set-up has been used to test other recent aspects of 
stochastic thermodynamics like the possibility to infer the 
potential V(x) from the measured stationary distribution 
and current [35] or a generalization of the Einstein relation 
beyond the linear response regime [36,37]. 



5.3 Transitions between different NESSs 

Just as a system can be driven from one equilibrium state 
to another by a time dependent potential V(x, A(r)), one 
can generate a transition from one NESS characterized by 
fi(x) to another one at f2(x) by a time-dependent force 
/(t). For the thermodynamic analysis of such a transition, 
it has become convenient to split the dissipated heat quite 
generally into two contributions, 

Aq ee Aq hk + Aq cx (36) 

where the house keeping heat 

q hk EE f dTi(T)/i-V(a;(T)) (37) 
Jo 

is the heat inevitably dissipated in maintaining a NESS 
[38,39]. For a transition from one NESS to another, one 
has both the IFT for the house keeping heat [40] 

(exp[-g hk /T]) = 1 (38) 

and the Hatano-Sasa relation [39] 

(exp-[ qex /T + A<f>(x,\)]) = l. (39) 

The latter relation allows one to obtain Clausius-type in- 
equalities for the minimal entropy change related to tran- 
sitions between different NESSs. 

Experimentally, the Hatano-Sasa relation has been ver- 
ified for a colloidal particle pulled through a viscous liq- 
uid at different velocities which corresponds to different 
steady states [41]. 



6 General stochastic dynamics 

6.1 Entropy production 

For the mechanically driven systems described so far the 
identification of a first law is simple since both internal 
energy and applied work are rather clear concepts. On 
the other hand the proof of both the IFT and the DFT 
shows that the first law does not crucially enter. In fact, 
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the proof of these theorems exploits only the fact that un- 6.2 Two classes of systems 

der time-reversal entropy production changes sign. If the 
changes in the medium entropy are written in the form 
(9, 10) only transition rates enter. Likewise, the identifi- 
ciation of system entropy requires only a solution of the 
Fokker-Plank equation. Therefore, similar relations should 
hold for a much larger class of stochastic dynamic models 
without reference to a first law. 

Indeed, one can derive these relations for stochastic dy- 
namics on an arbitrary set of states {n} where transitions 
between states m and n occur with a rate w mn (X), which 
may depend on an externally controlled time-dependent 
parameter A(r). The master equation for the time-dependent 
probability p„(r) then reads 



For a discussion, it is appropriate to distinguish two classes 
of systems based on whether or not their stationary distri- 
bution p s n for fixed A obeys the detailed balance condition 



p s n (X)w nm (X) = p s m (X)w mn (X). 



(45) 



6.2.1 Detailed balanced systems 



d T Pn(T) = I 

(AK(r)]. (40) 

The analogue of the fluctuating trajectory x(t) in the 
mechanical case becomes a stochastic trajectory n(r) that 
starts at no and jumps at times Tj from nj to n+ ending 
up at n t . As entropy along this trajectory, one defines [2] 



s(t) = -lnp n(r) (r) 



(41) 



where p n ( T )( T ) is the solution p n (r) of the master equation 
(40) for a given initial distribution p n (0) taken along the 
specific trajectory n(r). As above, this entropy will depend 
on the chosen initial distribution. 

The entropy s(r) becomes time-dependent due to two 
sources. First, even if the system does not jump, p n ^{r) 
can be time-dependent either for time-independent rates 
due to possible relaxation from a non-stationary initial 
state or, for time-dependent rates, due to the explicit time- 
dependence of p„(r). Including the jumps, the change of 
system entropy reads 



s(r) = 



drPn(r)(T) 



Pn(r)(r) 

stot(r) - s m (r). 



E 

3 



5{T-Tj)\n- 



(42) 
(43) 



where we define the change in medium entropy to be 

w - + 

s m(T )=^( r - Tj )ln-^. (44) 

i W n + n. 

3 3 J 

For a general system, associating the logarithm of the 
ratio between forward jump rate and backward jump rate 
with an entropy change of the medium seems to be an 
aritrary definition. Two facts motivate this choice. First, 
it corresponds precisely to what has been identified as ex- 
changed heat in the mechanically driven case (9,10). Sec- 
ond, one can easily show [2] that with this choice the total 
entropy production Z\s to t fulfills both the IFT (20) for ar- 
bitrary initial condition, arbitrary driving and any length 
of trajectory and the DFT (35) in the steady state, i.e. 
for constant rates. Of course, in a general system, there is 
no justification to identify the change in medium entropy 
with an exchanged heat. 



System which obey detailed balance formally resemble me- 
chanically driven systems without non-conservative force 
since for the latter, at fixed potential, detailed balance 
holds as well. Exploiting this analogy, one can assign a 
(dimcnsionlcss) internal "energy" 

e n (A) = -lnK(A) (46) 
to each state. The ratio of the rates then obeys 



w„ 



i(A) 



>(A) 



exp[e„(A) - e m (A)] 



(47) 



which looks like the familiar detailed balance condition in 
equilibrium. For time-dependent rates Wmn(A(r)), one can 
now formally associate a first law along the trajectory as 
follows. The analogue of the work is written in the form 

rt w - + 

w = d x e n (X(t)) = Vln " J " J + e„ (t) (A t ) - e„( )(A ) 

JO a W n+n- 

3 j j 

= Aq + AV (48) 

where the second lines identifies the "heat" with the medium 
entropy change and the last term represents a change 
in "internal energy". Even though one should not put 
physical meaning into these definitions for an abstract 
stochastic dynamics, the analogy helps to see immediately 
that the fluctuation relations quoted above for zero non- 
conservative force hold for these more general systems as 
well. Specifically, one has the "generalized" JR (22) and 
the corresponding end-point relation (30) with AT = 
and T = 1 . Similarly, with the identification 



E 

3 



[e„. + (tj) - e n .-(Tj)} - [e„. + (0) - e„._ (0)] (49) 



both (23) and (31) with T = 1 hold for such a master 
equation dynamics. The initial state in all cases is the 
steady state corresponding to Xq. 

The simplest realization of such a detailed-balanced 
system is any two-level system with time-dependent rates. 
In recent experiments [42,43], an optically driven defect 
center in diamond has been used to test the IFT for to- 
tal entropy production and the analogy of the Jarzynski 
relation for such a general stochastic dynamics [44]. 
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6.2.2 Unbalanced systems 

The more interesting class of networks, however, are those 
for which at constant A the detailed balance condition (45) 
is not fulfilled. Within the mechanical analogy, they cor- 
respond to systems with non-vanishing non-conservative 
force / ^ 0. For a mapping to this case, one would have 
to find the analogy of the conservative potential and the 
force / which is responsible for breaking detailed balance. 
The apparent arbitrariness of such a division is resolved 
by forming the analogy of the house-keeping heat (37) as 

P s -w n - n + 

ghkH^)] =]Tm " 3 1 3 . (50) 

P + w n + n ~ 

3 nj n j n i 

For detailed balanced systems, this expression vanishes 
identically. With this identification, it is clear that the 
IFT for the housekeeping heat (38) and the Hatano-Sasa 
relation (39) hold as well using these definitions (and T = 
1). For further fluctuation relations in such systems, see 
the comprehensive review [22]. 

7 (Bio)chemically driven systems 

Biochemical reactions constitute another important class 
of systems for which an embedding heat bath provides the 
source of stochastic dynamics. Therefore, the concepts re- 
viewed so far are ideally suited to develop a stochastic 
thermodynamics of such systems. Conceptually, two situ- 
ations should be distinguished described here only briefly. 

7.1 A single enzym or motor 

An enzym or molecular motor can be considered as a sys- 
tem which stochastically undergoes transitions from one 
state to to another state n. In such a transition, a chemi- 
cal reaction may be involved like hydrolysis which trans- 
forms one molecule ATP to ADP and a phosphate. These 
three molecular species are externally maintained at non- 
equilibrium conditions thereby providing a source of chem- 
ical energy, i.e., chemical work, to the system. In each 
transition, this work will be transformed into mechanical 
work, dissipated heat, or changes in the internal energy 
(with any combination of positive and negative contribu- 
tions). The formalism of stochastic thermodynamics al- 
lows to identify work, heat, and internal energy for each 
single transition [33,45] in close analogy to the mechani- 
cally driven case, thus illustrating the concepts described 
above for a more general stochastic dynamics. 

7.2 Chemical reaction networks 

Chemical reaction networks consist of a number of possi- 
ble reactions taking place between different types of reac- 
tants. The rates for each reaction are proportional to the 
number of possible reaction partners thus involving the 



stochiomctric coefficients. In order to describe such a net- 
work under non-equilibrium conditions, one should sepa- 
rate the reactants into two classes. The first class com- 
prise species whose concentration, i.e., chemical poten- 
tials, are fixed externally at constant values (like the ATP 
concentration in the cell). The second species comprises 
molecules whose number is traced along a trajectory. Un- 
der such conditions, chemical work, dissipated heat and 
changes in iternal energy can be identified on the level of 
a single reaction trajectory for which the general formal- 
ism yields IFTs, generalization of the Jarzynski relation 
and other relations [46]. So far, no experiments illustrating 
these concepts seem to have been reported. 



8 Stochastic dynamics of fields 

The Langevin dynamics discussed for the mechanically 
driven case can easily be generalized to N coupled de- 
grees of freedom [33,47]. A further generalization to the 
dynamics of stochastic fields, which as an exciting per- 
spective looks obvious, seems not yet to been explored 
systematically. 

We assume a dynamical equation for a field 0(r, r) with 
r e M. d of the type 

9 r 0(r,r)=F[0(r,r)]+C(r,r) (51) 

where F[cf>(r,T)] is an arbitrary possibly non-linear func- 
tional and C(r, t) Gaussian white noise of strength 

(C(r, r)C(/, r')) = 2D(r - r')6(r - r') (52) 

with a spatial correlation D(r — r'). Motivated by the me- 
chanically driven case, one defines a change in medium 
entropy 

As m = J dr J dr J dr' d T <j>{r, r)^" 1 ^ - r')F[<f)(r', r)]. 

(53) 

and a change in entropy of the field 

Z\ S = -ln^(r,T),T)|* (54) 

where p(<j){r, r), r) is the evolving probability distribution 
for the field. 

The validity of the IFT for total entropy production 
(20) for arbitrary initial state and arbitrary length t of the 
trajectory is obvious. If the dynamics settles into a steady 
state p s (<j>(r,T)), the DFT (35) also holds. 

As an illustrative example consider the KPZ equation 

d T h = \(Wh) 2 + vV 2 h + ( (55) 

for the height field /i(r, t) of a growing interface where A 
measures the strength of the non-linearity and v corre- 
sponds to a surface tension [48] . The white noise is uncor- 
rated in space and time (C(r, r)£(r', t')) = 2S(r — r')5(r — 
t') (setting I? to 1 for notational simplicity). 
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In the steady state, the total entropy production be- 
comes 



As t 



x L dT I dr ( <9r/ ^ v/i ) 2 

(i//2) J dr{Vh) 2 +\np s {h) 



(56) 



In one dimension, the known stationary distribution p s (h) <-• 
exp[— (is/2) J dr (V/i) 2 ] cancels the boundary term of the 
medium entropy so that the total entropy production over 
a time t along a trajectory in the steady state is given ex- 
actly by 



As tot = dr J dr{d T h){Vhf 



(57) 



The DFT (35) now makes a statement about the prob- 
ability distribution of the time integral of this particular 
product involving three powers of the field. By expanding 
the exponent, one can generate relations between particu- 
lar three point, six point, nine point functions and so on. 
While the immediate benefit of such relations is not obvi- 
ous, it looks worthwile to explore the consequences of this 
symmetry for correlations functions. 

In higher dimensions, the stationary distribution is un- 
known and the exact statement holds only for the sum of 
the three terms in (56). Clearly, only the term in the first 
line is extensive in time so one might expect that focussing 
on this term in the long time limit may be admissible but 
one should be aware of corrections arising from the two 
boundary terms. In any case, one could speculate whether 
the DFT restricts exponents or scaling functions if a suit- 
able scaling ansatz for correlation functions is put into it. 

If useful information can be extracted from such an 
approach for the KPZ equation, one might afterwards be 
tempted to look at other stochastic field equations. A 
particularly intriguing case are the Navier Stokes equa- 
tions with noise terms used to stochastically drive the 
system on large scales to model turbulence. In fact, for 
two-dimensional turbulence such an approach has been 
applied to the enstrophy cascade [49] . 



9 Concluding remark 

The notion "stochastic thermodynamics" had been intro- 
duced two decades ago for an interpretation of chemical 
reaction networks in terms of thermodynamic notions on 
the ensemble level [50]. From the present perspective, it 
seems even more appropriate to use this term for the re- 
fined description along the fluctuating trajectory. Both 
for mechanically and chemically driven systems in a sur- 
rounding heat bath, the thermodynamic concepts can lit- 
erally and consistently be applied on this level. As a gen- 
eralization to arbitrary stochastic dynamics, analogues of 
work, heat and internal energy obey similar exact rela- 
tions which ultimately all arise from the behaviour of the 
dynamics under time-reversal. How much closer such an 



approach can lead us towards a systematic understanding 
of non-equilibrium phenomena in general is a question too 
early to be answered yet. 
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